Unsupervised Time Series Extraction from Controller Area Network Payloads

This paper introduces a method for unsupervised tokenization of Controller Area Network (CAN) data payloads using bit level transition analysis and a greedy grouping strategy. The primary goal of this proposal is to extract individual time series which have been concatenated together before transmission onto a vehicle's CAN bus. This process is necessary because the documentation for how to properly extract data from a network may not always be available; passenger vehicle CAN configurations are protected as trade secrets. At least one major manufacturer has also been found to deliberately misconfigure their documented extraction methods. Thus, this proposal serves as a critical enabler for robust third-party security auditing and intrusion detection systems which do not rely on manufacturers sharing confidential information.Comment: 2018 IEEE 88th Vehicular Technology Conference (VTC2018-Fall

investr: An R Package for Inverse Estimation

Inverse estimation is a classical and well-known problem in regression. In simple terms, it involves the use of an observed value of the response to make inference on the corresponding unknown value of the explanatory variable. To our knowledge, however, statistical software is somewhat lacking the capabilities for analyzing these types of problems. In this paper, we introduce investr (which stands for inverse estimation in R), a package for solving inverse estimation problems in both linear and nonlinear regression models

Survival Analysis for Nanosatellites and Picosatellites

The nascent field of fractionated satellite architectures provides an opportunity to improve spacecraft modularity and afford greater flexibility, adaptability, and upgradeability to spacecraft constellations. Satellite modules within a coherent formation can be replaced without facing the challenges of manufacturing, assembly, or disassembly in the harsh space environment (e.g., satellite modules conducting electromagnetic formation flight (EMFF) are not physically connected such that one module may be replaced with potentially less risk of damaging or degrading the performance of the other modules). Conventionally, the depot for constellation replenishment is located on Earth, however, minor augmentations to spacecraft formations cannot be conducted economically under such a framework. The present research proposes the utilization of proactively launched supply depots to replenish geostationary formations from ultrageostationary orbit (i.e., that volume of space encompassed between the altitude of geostationary orbit and the altitude of the L1 Lagrange point). This work explores reliability factors associated with such a concept by conducting a survival analysis for nanosatellites and picosatellites. Time to failure data is collected for 85 spacecraft in the nano- (1.01 – 10 kg wet mass) and pico- (0.11 – 1 kg wet mass) classes without data censoring. These spacecraft were launched between 2010 and 2019, inclusive, having an internationally diverse set of owners from the sectors of military, government, commercial, and academia. This data is used to build a distribution for the survival analysis of satellites in these classes. JMP Pro 13 is used to conduct a goodness-of-fit test for multiple distributions. Analysis (using a standard alpha value of 0.05) indicates that the data is from a two-parameter Weibull distribution wherein the spacecraft experience beneficial aging

The Effects of Individual Differences, Non‐Stationarity, and The Importance of Data Partitioning Decisions for Training and Testing of EEG Cross‐Participant Models

EEG-based deep learning models have trended toward models that are designed to perform classification on any individual (cross-participant models). However, because EEG varies across participants due to non-stationarity and individual differences, certain guidelines must be followed for partitioning data into training, validation, and testing sets, in order for cross-participant models to avoid overestimation of model accuracy. Despite this necessity, the majority of EEG-based cross-participant models have not adopted such guidelines. Furthermore, some data repositories may unwittingly contribute to the problem by providing partitioned test and non-test datasets for reasons such as competition support. In this study, we demonstrate how improper dataset partitioning and the resulting improper training, validation, and testing of a cross-participant model leads to overestimated model accuracy. We demonstrate this mathematically, and empirically, using five publicly available datasets. To build the cross-participant models for these datasets, we replicate published results and demonstrate how the model accuracies are significantly reduced when proper EEG cross-participant model guidelines are followed. Our empirical results show that by not following these guidelines, error rates of cross-participant models can be underestimated between 35% and 3900%. This misrepresentation of model performance for the general population potentially slows scientific progress toward truly high-performing classification models

A Proposed Methodology to Characterize the Accuracy of Life Cycle Cost Estimates for DoD Programs

For decades, the DoD has employed numerous reporting and monitoring tools for characterizing the acquisition cost of its major programs. These tools have resulted in dozens of studies thoroughly documenting the magnitude and extent of DoD acquisition cost growth. Curiously, though, there have been extremely few studies regarding the behavior of the other cost component of a system\u27s life cycle: Operating and Support (O&S) costs. This is particularly strange considering that O&S costs tend to dominate the total life cycle cost (LCC) of a program, and that LCCs are widely regarded as the preferred metric for assessing actual program value. The upshot for not examining such costs is that the DoD has little knowledge of how LCC estimates behave over time, and virtually no insights regarding their accuracy. In recent years, however, enough quality LCC data has amassed to conduct a study to address these deficiencies. This paper describes a method for conducting such a study, and represents (to the authors’ knowledge) the first broad-based attempt to do so. The results not only promise insights into the nature of current LCC estimates, but also suggest the possibility of improving the accuracy of DoD LCC estimates via a stochastically-based model

Deep Long Short-term Memory Structures Model Temporal Dependencies Improving Cognitive Workload Estimation

Using deeply recurrent neural networks to account for temporal dependence in electroencephalograph (EEG)-based workload estimation is shown to considerably improve day-to-day feature stationarity resulting in significantly higher accuracy (p \u3c .0001) than classifiers which do not consider the temporal dependence encoded within the EEG time-series signal. This improvement is demonstrated by training several deep Recurrent Neural Network (RNN) models including Long Short-Term Memory (LSTM) architectures, a feedforward Artificial Neural Network (ANN), and Support Vector Machine (SVM) models on data from six participants who each perform several Multi-Attribute Task Battery (MATB) sessions on five separate days spread out over a month-long period. Each participant-specific classifier is trained on the first four days of data and tested using the fifth’s. Average classification accuracy of 93.0% is achieved using a deep LSTM architecture. These results represent a 59% decrease in error compared to the best previously published results for this dataset. This study additionally evaluates the significance of new features: all combinations of mean, variance, skewness, and kurtosis of EEG frequency-domain power distributions. Mean and variance are statistically significant features, while skewness and kurtosis are not. The overall performance of this approach is high enough to warrant evaluation for inclusion in operational systems

Taming the Hurricane of Acquisition Cost Growth – Or At Least Predicting It

Cost growth is a persistent adversary to efficient budgeting in the Department of Defense. Despite myriad studies to uncover causes of this cost growth, few of the proposed remedies have made a meaningful impact. A key reason may be that DoD cost estimates are formulated using the highly unrealistic assumption that a program’s current baseline characteristics will not change in the future. Using a weather forecasting analogy, the authors demonstrate how a statistical approach may be used to account for these inevitable baseline changes and identify related cost growth trends. These trends are then used to reduce the error in initial acquisition cost estimates by over one third for major defense acquisition programs, representing a more efficient allocation of $6 billion annually

Extending Critical Infrastructure Element Longevity using Constellation-based ID Verification

This work supports a technical cradle-to-grave protection strategy aimed at extending the useful lifespan of Critical Infrastructure (CI) elements. This is done by improving mid-life operational protection measures through integration of reliable physical (PHY) layer security mechanisms. The goal is to improve existing protection that is heavily reliant on higher-layer mechanisms that are commonly targeted by cyberattack. Relative to prior device ID discrimination works, results herein reinforce the exploitability of constellation-based PHY layer features and the ability for those features to be practically implemented to enhance CI security. Prior work is extended by formalizing a device ID verification process that enables rogue device detection demonstration under physical access attack conditions that include unauthorized devices mimicking bit-level credentials of authorized network devices. The work transitions from distance-based to probability-based measures of similarity derived from empirical Multivariate Normal Probability Density Function (MVNPDF) statistics of multiple discriminant analysis radio frequency fingerprint projections. Demonstration results for Constellation-Based Distinct Native Attribute (CB-DNA) fingerprinting of WirelessHART adapters from two manufacturers includes 1) average cross-class percent correct classification of %C \u3e 90% across 28 different networks comprised of six authorized devices, and 2) average rogue rejection rate of 83.4% ≤ RRR ≤ 99.9% based on two held-out devices serving as attacking rogue devices for each network (a total of 120 individual rogue attacks). Using the MVNPDF measure proved most effective and yielded nearly 12% RRR improvement over a Euclidean distance measure

Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks

Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance

Uncertainty Evaluation in the Design of Structural Health Monitoring Systems for Damage Detection

The validation of structural health monitoring (SHM) systems for aircraft is complicated by the extent and number of factors that the SHM system must demonstrate for robust performance. Therefore, a time- and cost-efficient method for examining all of the sensitive factors must be conducted. In this paper, we demonstrate the utility of using the simulation modeling environment to determine the SHM sensitive factors that must be considered for subsequent experiments, in order to enable the SHM validation. We demonstrate this concept by examining the effect of SHM system configuration and flaw characteristics on the response of a signal from a known piezoelectric wafer active sensor (PWAS) in an aluminum plate, using simulation models of a particular hot spot. We derive the signal responses mathematically and through the statistical design of experiments, we determine the significant factors that affect the damage indices that are computed from the signal, using only half the number of runs that are normally required. We determine that the transmitter angle is the largest source of variation for the damage indices that are considered, followed by signal frequency and transmitter distance to the hot spot. These results demonstrate that the use of efficient statistical design and simulation may enable a cost- and time-efficient sequential approach to quantifying sensitive SHM factors and system validation